SPECIFICATION 



TITLE OF THE INVENTION 

SPEECH ^INTERACTION TYPE ARRANGEMENTS 

FIELD OF THE INVENTION 

The present invention relates to a speech interaction 
arrangement (terminals, methods, implementations) for 
helping control simultaneous execution of a plurality of 
applications . 

BACKGROUND 

When a plurality of applications exist in one system, 
the applications may be able to be simultaneously executed. 
However from the viewpoint of a user, the number of the 
applications the user can simultaneously execute may be 
extremely small, in most cases, one. Particularly when 
operations by speech interaction are executed, the number 
of applications simultaneously interacting with the user is 
preferably limited to one. 

On the other hand, when the user desires to operate 
an application other than the application currently 
interacting with the user, the user in a speech interactive 
system may speak a global command corresponding to a 
user-desired application. Then the system transfers to a 



state of interaction between the user and the user-desired 
application, and to a state of execution of a specific 
function. It is therefore important for the user's 
satisfaction that the user is able to directly operate the 
5 user-desired application by means of speech without using 
a textual menu each time to select the user-desired 
application. 

One background interaction type user interface unit 
detects that a content of a user's speech corresponds to an 

10 application "other than one being currently executed, and 
executes the speech interaction application corresponding 
to the speech content instead of the currently interacting 
one. (for example, see JP-A No. 56694/2001) . 

Car navigation systems may be sold by their 

15 manufacturers in the form that all programs to be executed 
in the system are previously installed in the system. In this 
case, all applications are known to the manufacturer, and 
thus any state of the system could be anticipated and 
adjusted to execute normal interaction. 

20 Current speech recognition systems (e.g. car 

navigation system) are not perfect, for example, a 
recognition ratio may be decrease as the number of commands 
to be recognized is increased. As one example, when terminal 
functions are advanced and more applications are installed, 

25 the number of the commands to be managed is greatly 
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increased. Then, the increased command vocabulary may 
lowers the recognition ratio. 

Car navigation systems where applications can be 
installed from the Internet via mobile phones are possible. 
5 In this case, which types of applications that are installed 
within a system at any given time may not be assumed. 
Therefore, global commands to be recognized cannot be 
optimally set according to all the possible states of the 
navigation system by previously counting the states, for 
-10 example, the number of the commands to be managed cannot be 
decreased by adjusting the applications. 

BRIEF DESCRIPTION OF THE DRAWINGS 

Fig. 1 is a block diagram explaining .one example 
15 embodiment of an interaction type terminal; 

Fig. 2 explains one example embodiment of a management 
table of global commands; 

Fig. 3 is an example flowchart explaining one example 
embodiment of processing a result of speech recognition; 
20 Fig. 4 explains one example embodiment of a management 

table of global commands ; 

Fig. 5 is a block diagram explaining one example 
embodiment of an interaction type terminal; 

Fig. 6 is a block diagram explaining one example 
25 embodiment of an interaction type terminal; 
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Fig. 7 explains one example embodiment of 
classification of global commands; 

Fig. 8 is an example flowchart explaining one example 
embodiment of the process when an application is downloaded; 
5 Fig. 9 explains another example embodiment of 

classification of global commands; 

Fig. 10 explains one example embodiment of an example 
global command management table which can be customized by 
a user; 

10 Figs. 11A and 11B explain one example embodiment of 

a screen for editing global commands ; 

Fig. .12 explains another example embodiment of a 
screen for editing global commands; 

Fig. 13 is another example block diagram explaining 
15 another example embodiment of an interaction type terminal ; 

Fig. 14 explains one example embodiment of the 
information contained in an application; 

Fig. 15 explains another example embodiment of the 
information contained in an application; 
20 Fig. 16 is a example block diagram explaining one 

example embodiment of a method for providing a speech 
interaction application; 

Fig. 17 explains an example embodiment of a screen 
display of a car navigation system; 
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Fig. 18 explains one example embodiment of a global 
command list of a car navigation system; 

Fig. 19 explains one example embodiment of a global 
command list of each application; and 

Fig. 20 explains one example embodiment of a change 
of the global command list when an application changes its 
state. 

DETAILED DESCRIPTION 

Before beginning a detailed description of the 
.subject invention, mention of the following is in order. 
When appropriate, like reference numerals and characters 
may be used to designate identical, corresponding or similar 
components in differing figure drawings. As manufacturing 
techniques mature over time, it is expected that devices, 
apparatus, etc., of smaller size, improved quality, 
additional feature etc. could be manufactured. With regard 
to description of any flowchart it might possible to change 
an order of steps. As a final note, well known components 
for system (e.g. car navigation) may not be shown within the 
FIGS, for simplicity of illustration and discussion, and so 
as not to obscure the invention. Further, arrangements may 
be shown in block diagram form in order to avoid obscuring 

4 

the invention, and also in view of the fact that specifics 
with respect to implementation of such block diagram 
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arrangements are highly dependent upon the. platform within 
which the present invention is to be implemented, i.e. , such 
specifics should be well within purview of one skilled in 
the art. In other instances, detailed descriptions of 
5 well-known methods and components are omitted so as not to 
obscure the description of the invention- with 
unnecessary/excessive detail. Where specific details 
(e.g., systems, flowcharts) are set forth in order to 
describe example embodiments of the invention, it should be 

10 apparent to one skilled in the art that the invention can 
be practiced without, or with variation of, these specific 
details. Finally, it should be apparent that differing 
combinations of hard-wired circuitry and software 
instructions can be used to implement embodiments of the 

15 present invention, i.e., the present invention is not 
limited to any specific combination of hardware and 
software. 

Although main example embodiments of the present 
invention will be described using an example system block 

20 diagram in an example terminal environment, practice of the 
invention is not limited thereto, i.e. , the invention may 
be able to be practiced with other types of electronic 
devices (e.g., personal computers, servers) and/or systems 
(mainframes), and in other types of environments (e.g., 

25 networks) . 
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Example embodiments of the present invention 
disclosed herein are explained in detail below, although 
practice of present invention is not limited to such example 
embodiments . 

5 Global commands are used for making an application 

currently interacting with a user, and applications other 
than the currently-interacting application, to execute some 
predetermined operation. When the user desires to operate 
an application not currently interacting with the user, the 

10 user speaks a command corresponding to the user-desired 
application, so that, for example, a state of a system can 
be transferred to interaction between the user and the 
application. As required, the global commands may mean 
internal expressions in a terminal and a server of the 

15 commands . 

Next, example embodiments of the present invention 
are explained with reference to the drawings. 

Fig. 1 is one example embodiment of an interaction 
type terminal. A terminal 10 comprises a speech input unit 
20 11, a speech recognition engine 12, a memory 13, a control 
unit 14, and one or more applications 15a to 15c. 

The terminal may be installed as part of a system shown 
representatively by a block 16. For example block may 
represent a vehicle, a portable computer and a room, etc. 
25 Although the present invention is not limited thereto, the 
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following discussions may use installation within a vehicle 
system (e.g. car) as an example for discussion. 

The speech input unit 11 is , for example, a microphone 
attached to a sun visor of an automobile. The speech 
5 recognition engine 12 recognizes both global commands 

recorded in the memory 13 and specified by the control unit 
14 , and also local commands required for interaction between 
the application 15 and the user. 

The control unit 14 receives a result of the speech 

10 recognition, and provides control to execute the 

application 15 corresponding to the global command. To make 
the global, commands correspond to the applications 15, ID 
information for identifying the applications 15a to 15c may 
be associated with the respective global commands. The 

15 control unit 14, according to change of a state of each 
application 15, may specify any one of global commands to 
be recognizable by the speech recognition engine 12. In 
Fig.l the control unit 14 instructs the speech recognition 
unit 12 directly, but it may be possible to instruct the 

20 speech recognition unit 12 through record unit 13. 

By receiving, from the applications, reports about 
change of states of their operations, the control unit may 
monitor the states of the applications, and adjust a list 
of recognizable commands accordingly . Or as shown in Fig. 13 , 

25 a discrete monitor unit 134 may be provided to monitor 
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applications 133(a,b,c). When states of operations of the 
applications 133 change, the monitor unit 134 may report the 
change to the control unit 131. With such a configuration, 
the applications 133 do not need to have a system to report 
5 states to the control unit 14. 

The global command management used by the control unit 
may be as follows. For example, the record unit 13 may record 
and^ manage a table such as shown in ,for example, Fig. 2, 
where each global command may have stored there with a 

10 corresponding application ID, application states in which 
the global commands become valid, a flag representing 
whether the global commands are presently valid, and so on. 
When each of the applications 15 is activated, stopped, or 
has the right to preferentially interact with the user, it 

15 may report its states to the control unit 14, which may 
rewrite appropriate flags. 

For example, when an application is not operating, a 
name of the application may become valid as a global command 
for activation of the application. After the activation, a 

20 plurality of operation commands may become valid (active) 
to achieve more concrete operations, and the name of the 
application which has previously been become valid may 
become invalid (inactive i.e. presently unrecognizable) as 
a global command. 
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There may be many example states of operations of each 
application. For example, the states that the application 
is installed but not activated; that the application is 
activated but not interacting with the user; and, that the 
5 application is interacting with the user. Depending on the 
states, required global commands may be greatly different, 
and these states may be universal among applications 
regardless of a type of application. Therefore, the global 
commands may be managed according to change of such 

10 operation states. 

The global commands may be described using SRGF 
(Speech Recognition Grammar Format) , which is standard as 
a grammar for speech recognition. In this case, direct 
grammar description, or the name of a file in which grammar 

15 description is made, may be described. 

Fig. 3 is an example control flow of this terminal. 
After speech recognition (30) , it is checked whether a 
result of the speech recognition is normal (31) . When the 
result is not normal, error processing is performed (35). 

20 As an example embodiment of error processing, a guidance 
prompting the user to speak may be outputted to return to 
the process 30 again. When the result is normal, it is judged 
whether the result matches a local command which is an 
interaction command corresponding to a 

25 currently-interacting application (32) . When the result 
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matches a local command, the process corresponding to the 
local command is processed for a currently-interacting 
application (36) . When the result does not match a local 
command, the result may be judged as a global command, and 
5 a table as shown in Fig. 2 is referenced to obtain IDs of 
all applications corresponding to the global command (33) . 
Then, the applications corresponding to the obtained IDs are 
instructed to execute processes corresponding to the 
recognized global command (34) . 
10 Through such a flow, when there is the same command 

in both the local commands and global commands, the local 
command may be preferentially executed. As a result, when 
an interaction command corresponding to a 

currently-interacting application is the same as a global 
15 command of a different application, interaction with the 
currently-interacting application can be continued without 
spoiling a flow of the interaction. 

To execute an actual process for a global command 
spoken by the user, process information is required for each 
20 global command. Information representing functions 

corresponding to the global commands may be stored, for 
example, in the memory 13, or in the applications 15. A table 
as shown in Fig. 14 may be added to each application 15 , where 
a current state of each application 15 and the information 
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representing functions corresponding to commands spoken by 
the user can be recorded. 

In the above-described configuration, the control 
unit may transmit an execution instruction to an application 
5 corresponding to an inputted global command, so that a 
corresponding appropriate process is executed. 

Since the. number of possible states of each 
application is smaller than that of the whole system, the 
management of global commands may be efficient by 
10 corresponding the states of each application to the global 
commands to be recognized. Changing global commands 
recognizable according to the states of the applications may 
decrease the number of commands to be recognized at a time. 
As a result, an amount of the processes can be decreased, 
15 and a command not required by the situation is prevented from 
being wrongly recognized and executed. 

The correspondence between the global commands and 
applications 15 may not be limited to one-to-one 
correspondence. More particularly one application 15 can 
20 correspond to a plurality of global commands. One global 
command can correspond to a plurality of applications 15, 
so as to execute simultaneously a plurality of the 
. applications 15 in response to one speech by the user. In 
an example shown in Fig. 2, a command "search surrounding 
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information'' corresponds to applications 15 having IDs 2 and 
5. 

As described-above, the applications 15 may include 
a table such as shown in Fig. 14, so that each application 
5 15 may execute a different function even when one global 
command corresponds to a plurality of the applications 15. 
For example, when a global command "setting destination'' is 
spoken, and when the command corresponds to a destination 
setting application and traffic information application, 

10 the destination setting application may start to interact, 
with the user, and the traffic information application may 
obtain surrounding traffic information to be on standby. 

Even when different applications have the same global 
command, if the global command corresponds to the state of 

15 a currently-interacting application, only the application 
15 currently interacting with the user "may execute a process 
reguired by the user. For example in Fig. 4, a command "help" 
corresponds to application IDs 1 and 3. However, only the 
application 15 having the ID 1 is currently interacting, 

20 i.e. only a flag of a global command corresponding to the 
ID 1 is valid. Therefore, when the user speaks "help", a help 
process of the application 15 having the ID 1 is executed. 

With the above-described method, an application not 
required by the user may be prevented from being executed 
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because of a collision of global commands such as "menu" and 
"help", generally used in applications. 

Another example method for avoiding such collision is 
as follows. In the case that the user speaks a global command 
5 shared by a plurality of applications, it may be 
predetermined which application executes a process 
corresponding .to the command. A criterion for determining 
the application to be executed is such that the applications 
are prioritized to execute a highest prioritized 

10 application. In this case, the priority order does not 

necessarily need to be fixed, e.g., may be dynamic and change 
over time. For example the application currently 
interacting with the user may be prioritized higher, or an 
application presently displaying a button on a screen 

15 corresponding to the global command may be prioritized 
higher. 

Instead of a dynamic method for assigning priorities 
to protect against collision, a database where all global 
commands of all applications are registered may be provided 

20 to a server to help recognize and avoid collision. When a 
new application is added, it may be checked whether its 
global commands collide with the other registered global 
commands. Also to avoid the situation that a general command 
(such as "menu") is used for only one application, a 

25 registrant may be warned about the collision to voluntarily 
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change a name of the command at the time when the collision 
becomes a problem. In such a method, the warning may be given 
not only when the global commands perfectly match, but also 
when there may be a global command having a pronunciation 
which is similar to those of other global commands and which 
may thereby tend to be wrongly recognized. To judge a 
pronunciation similarity, the method of calculating a 
distance of Roman alphabets with which each global command 
is described by Dynamic Programming and a penalty of 
replacement, elimination, and insertion of each alphabet is 
added, may be adopted. 

In the configuration of the present invention, not 
only local commands but also global commands may be used for 
a currently-interacting application. The 

currently-interacting application 15 may preferably prompt 
the next speech of the user, for example, by providing a 
guidance to the user (e.g. by presenting command choices by 
synthesized speech) . And the local commands may then be used 
to preferably to recognize the prompted speech by the user. 
On the other hand, regardless of the prompt, the user may 
call a help function when puzzled about the operation. The 
local commands constantly change according to a flow of 
interaction, but global commands, unrelated to flow of the 
interaction, may not frequently change. So it may be 



efficient that a help command may be managed as a global 
command . 

As described above, the example embodiments of the 
present invention may provide an interaction type terminal 
5 for managing global commands according to states of a 
plurality of applications in a system. 

In the above-described embodiments, only one memory 
13 for recording global commands and only one speech 
recognition engine 12 exist in the system. The present 

10 invention is not limited to such a configuration. For 

example, a terminal 50 of Fig. 5 may include a plurality of 
speech recognition engines 51(a,b,c) and memories 
53(a,b,c) , which may correspond to the applications 15. In 
this case, a control unit 52 may receive a plurality of. 

15 results of the recognition. The speech recognition engines 
51 may add score information to the results of the 
recognition to decide which result of the recognition is 
used by the control unit 52. 

With such a configuration, the recognition engines 51 

20 may be respectively specialized to types of global commands 
to be recognized, thereby increasing the process efficiency 
and recognition ratio. For example, the recognition engine 
51a may recognize continuous words, the recognition engine 
51b may recognize continuous numbers, and the recognition 

25 engine 51c may recognize geographic names. 
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Fig. 6 shows another example embodiment of the 
interaction type terminal. A terminal 60 may include the 
speech input unit 11 , a speech recognition 67, an input unit 
61, a memory 62, a communication unit 63, a control unit 65, 
5 and one or more applications 66(a r b,c). 

In an embodiment of Fig. 6, compared to the embodiment 
of Fig. 1,. the input unit 61 and communication unit 63 are 
added as components. With the communication unit 63, the 
terminal 60 can access an external server 64 to download 
10 (e.g. new or unprogramed) applications 66. A method based 
on, for example, OSGi (Open Services Gateway Initiative) may 
be used for the download. 

The speech recognition engine 67 and control unit 65 
may be configured as execution programs executed in the 
15 terminal 60. In this case, when these execution programs 
have been improved, the improved versions may be easily 
downloaded via the communication unit 63, and may replace 
the outdated ones, so that the terminal 60 is updated. 
Through such a procedure, the user may be 
20 advantageously enabled to download and use a new or improved 
application 66 produced after the user purchases the 
terminal, and/or a new improved speech recognition engine 

67. With such an update function, the user may 

> 

advantageously not need to purchase terminals frequently. 
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The downloaded application 66 may preferably include 
global command information, so that, like the previously 
installed applications 66, the downloaded application 66 
may be operated by speech interaction. 

In this case, for example, as shown in Fig. 7, the 
global command information may be classified according to 
the states of the applications 15, and stored, for example, 
in information headers of the application programs. 

Fig. 8 is one example of a process flow of the terminal 
when the application 66 is downloaded. The control unit 65 
may monitor whether an application 66 is being downloaded 
through communication 63 (80) . For example, download may be 
executed from a server according to the user's request 
instruction via input unit 61 of the terminal, or 
automatically executed from the server. 

When the ■ application 66 is downloaded, the global 
command information contained in the application 66 may be 
added to a management table as shown in Fig. 2 (81) . Because 
the global command information contained in the application 
66 may not include information on whether each global 
command is valid or invalid, the management table may be so 
rewritten that all the added global commands are initially 
designated invalid (82) . After that, the management table 
may be selectively rewritten so that global commands may be 



classified into ones registered at the time of installation 
so as to become valid (83) . 

The storing of the global commands, which are for at 
the time of activation and at the time of interaction, in 
the record unit 62 may be done when the application is 
downloaded through above-described operations. And these 
global commands may be flagged to be recognizable when the 
application 66 reports a change of its state to the control 
unit 65. 

Fig. 16 is an example of a system configuration for 
providing a speech interaction application that executes 
the above-described example embodiment. A plurality of 
terminals 160(a,b,c) are connected to a server 162 over a 
network 161. In the server 162, a speech interaction 
application 163 may be recorded in, for example,, a HDD 
device. As explained in Fig. 5, the speech interaction 
application 163 may be related to the global commands 
classified according to the states of the applications, 
which global commands may be also recorded in, for example, 
a HDD device. When any terminal 160 requests a download of 
the speech interaction application 163 over the network 161, 
the server 162 may transmit, to the terminal 160, the speech 
interaction application 163 and the global commands related 
thereto . 



Next, an example embodiment of editing of global 
commands is explained. A function of an application to be 
used may differ according to its user, so that the commands 
to be used may depend on the user. When there is a command 
5 hardly used by the user in the commands registered at the 
time of activation, the user may desire to explicitly remove 
it from the commands to be recognized. Thereby decreasing 
the possibility of conflict and malfunction of the system. 

When the commands frequently used by only a few users 
10 are blindly classified into commands registered at the time 
of activation from the beginning, the number of the commands 
may become unnecessary excessive (i.e. 

over-comprehensive) , decreasing the recognition ratio. In 
this case, a class for user-customized global commands may 

15 be provided. The commands of this class may be assumed not 
to be recognized according to the states of the application 
66 in a default state, but to be explicitly set by the user 
as commands to be recognized. With the input unit 61, the 
user may edit a management table, as shown in Fig. 10, 

20 recorded in the memory 62 to decide whether each global 
command is to be recognized. A remote controller, a touch 
panel, and so on may be used as the input unit 61. 

Fig. 11 shows one example of a display screen of the 
terminal when the user edits global commands. On a screen 

25 (Fig. 11(A)) for editing global commands, the user may 
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select, by use of, for example, an up-and-down button of a 
remote controller, the application 66 whose global commands 
are to be edited, and may press a decision button to display 
a list of the global commands corresponding to the selected 
5 application 66 (Fig. 11(B)). Then the user may select a 
global command to be edited from the editable global 
commands by use of, for example, an up-and-down button of 
a remote controller, and press a decision button, whereupon 
"AUTO", -ON", and "OFF" may be switched in the "setting" 
10 field. 

When the global command is set as "AUTO", the global 
command may become valid or invalid according to information 
on "ON/ OFF" corresponding to each state; e.g. "activating", 
"executing", and "interacting" of a screen of Fig. 11. This 

15 is a default state. On the other hand, when the field 

"setting" is set as "ON", the global command may be always 
valid regardless of the state of the application 66. When 
the field "setting" is set as "OFF", the global command may 
be always invalid regardless of the state of the application 

20 66. 

In the above example, regardless of the state of the 
application 66, the validity of the global command which is 
set as "ON" or "OFF" by the user may be fixed. A flag 
corresponding to each state of the application may be 
25 changed using an interactive screen such as shown in Fig. 
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12. User may rewrite the "state" field of the management 
table as shown in Fig. 2, so that the user may decide the 
validity of a grammar for the states of the application. 
Additionally, a new command may be added and set. This 
5 achieves a customized terminal which reflects the user's 
taste . 

Fig. 13 shows another example embodiment of the 
interaction type terminal. A terminal 130 may have a 
configuration where an input unit 132 and monitor unit 134 
10 are added to the terminal 10 of Fig. 1. The input unit 132, 
which takes a form of, for example, a switch equipped to a 
steering, reports that the user desires to interact with a 
system. 

When the user starts to interact with a system, it may 
15 be efficient that the start of the interaction is reported 
to the system by use of some input. Because the user may not 
have knowledge about global commands, the system may provide 
a guidance to prompt the interaction when there is an input 
for the start of the interaction. In this case, the number 
20 of applications to start the interaction may be preferably 
fixed to one according to the situation, and may be, for 
example, an application which is in the state of 
"interacting". 

The control unit 131, according to an input from the 
25 input unit 132, may instruct the application in the state 
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of "interacting" such as an application 133a to start to 
interact with the user. After receiving the instruction, the 
application 133a may start the interaction, for example, by 
outputting a guidance prompting the user to speak. 
5 On the other hand, a user already accustomed to the 

system may desire quick handling of global commands by 
skipping the guidance- of the system. To skip the guidance, 
a speech for the global command may be detected. As a more 
reliable method, the user may give an input for skipping the 

10 guidance to the system, so that the guidance may stop, and 
a state for waiting a speech inputted by the user appear, 
increasing a satisfaction level of the user accustomed to 
the system. The input for skipping the guidance may be 
inputted, for example, through the input unit 132, shown in 

15 Fig. 13, which reports the start of interaction to the 

system. A different input unit may be provided. Two switches 
may be equipped to a steering wheel, for example, where one 
is used as the input unit 132. and another is used as the 
different input unit. 

20 This configuration may achieve a reliable process to 

avoid the possibility that a speech input may cause a 
malfunction, in an environment having a high noise level, 
e.g., such as in a running automobile. 

When the user speaks a global command to operate an 

25 application 133b other than the currently-interacting 
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Application 133a, the interaction may not always need to be 
transferred to the application 133b. When the application 
133b executes the user-desired operation, it may be better 
to continue the interaction with the application 133a. To 
response to such a situation, the applications 133 may have 
information on whether to start to interact with the user 
after executing a process corresponding to a command. In the 
present invention, information for achieving the above 
procedure, in other words, information on priority for 
determining whether the interaction is transferred to the 
application that has received a command instruction is 
called a priority level. 

Concretely, as shown in the example of Fig. 15, a table 
where priority level information is added to the table of 
Fig. 14 may be contained on each application. After a 
function corresponding to a global command spoken by the 
user is executed, it may be decided whether to interact with 
the user according to the priority level information. 

For example, it is assumed that the application 133b 
managing commands shown in Fig. 15 is in the activated state, 
and that the command "VOLUME UP" is registered as a global 
command of the application 133b and is valid. In this 
situation, when the command "VOLUME UP" is spoken, the 
control unit 131 may instruct the application 133b 
corresponding to the command "VOLUME UP" to execute the 
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command "VOLUME UP". The application 133b , referring to the 
table of Fig. 15, may execute an operation for turning up 
the volume by one step, which operation corresponds to the 
command u VOLUMEUP " . 
5 In this process, the application 133b may refer to the 

information on the priority level. In this case, the 
priority level is low, so that the application 133b 
continues the "activated" state, and does not interact with 
the user. Therefore, when the user reports the start of the 

10 interaction again by use of the input unit 132, the 
application 133a may start to interact with the user. 

On the other hand, in the above situation, when the 
command "SELECT" is spoken, the application 133b may refer 
to the priority level information. The priority level of the 

15 command "SELECT" is high, a state of the application may 
transfer to "interacting", and the control unit 131 receives 
the change of the state. Additionally, the application 133b 
may execute a process "DIALOG .(SELECT)". In this example, 
it is assumed that the interaction with the user is done for 

20 selecting a name of a tune. 

In this example, the information on the states of the 
applications 133- and the information on the priority levels 
may be recorded in the table, which is to be referenced. 
These pieces of information may change over time according 

25 to usage of the user. For example, when the present terminal 
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is a car navigation system, the table may not be referenced. 
Instead, when necessary, it may be instantaneously decided 
which application 133 enters the "interacting". state and 
whether to interact after a command is spoken according to, 
5 for example, information on the running state. 

Referring to Fig. 17, an embodiment when the present 
invention is applied to a car navigation system, which is 
an in-vehicle terminal, is explained. A car navigation 
system such as shown in Fig. 17 includes a display unit 171 4 

10 And as a car navigation system, any example of the example 
embodiment terminals shown in this specification may be 
applied. It is obvious that in addition to the elements shown 
in each terminal system, a car navigation may have its 
own-car location obtaining unit and memory storing map 

15 information therein or other devices connected via network. 

As shown in Fig. 17, the screen may be divided into 
two areas, one being a navigation screen 172, and another 
being a CD play screen 173. By using, for example, the OSGi 
platform, the car navigation system may execute 

20 simultaneously a plurality of applications. Each 

application may be displayed on the display area assigned 
thereto, so that the screen as shown in Fig. 17 is configured 
in a predetermined manner. In this example, a "navigation 
application" and "music application" are displayed on the 

25 screen. 
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Each application may not need to be always assigned 
a display area. For example, normally, an application for 
monitoring occurrence of accidents may not need to be 
displayed on the screen, so that the application may not be 
5 displayed on the screen. 

In the example embodiment of a navigation screen 
displayed on the screen 172, an own-car location icon 174 
may be displayed on a route according to. information 
obtained by own car location obtaining unit. Map information 

10 may be recorded on the terminal or obtained by accessing a 
server. This navigation application, which can interact 
using speech, may set a destination, for example, in 
response to an input of a command "setting destination", 
search a route from own car location to the destination in 

15 response to a command "search route", and display route 
information 175 on the map. 

Fig. 18 is a list of example global commands that may 
be received by the car navigation system in the state of Fig. 
17. In addition to the two applications displayed on the 

20 screens, this system has obtained a "traffic information 
application", so that a command corresponding to the 
"traffic information application" may also be managed. For 
example, when the application is downloaded from the server 
by use of the OSGi platform, a global command may be 

25 registered as a command for activating the application when 
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the application is installed. This achieves that an 
application not displayed on the screen may be executed by 
calling it directly by use of a speech command, instead of 
calling the menu using a remote controller, a touch panel, 
5 and so on. 

In a car navigation system having a fixed menu 
configuration, an application may be called while the user 
is driving the car by use of the remote controller by 
memorizing a location relationship of the menu. In a car 

10 navigation system where an application can be added, the 
menu configuration may change, so that the screen needs to 
be seen when operating the system with the remote 
controller. Safety operations while the user is driving the 
car may be achieved in the car navigation system where the 

15 menu configuration changes are able to be made by making 
speech input of commands . 

In the command list example shown in Fig. 18, the 
number of global commands listed as navigation applications 
is greater than that of music applications, and only one 

20 global command of a traffic information application is 

provided. This is because each application includes global 
command lists 191 to 193 as shown in Fig. 19, and receivable 
commands may switch according to a state of each 
application. In this example, the navigation application is 

25 in the "interacting" state, the music application is in the 
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"activated" state, and the traffic information application 
is in the "installed" state. Therefore, many commands for 
route guidance, such as "setting destination" and "scale up 
(screen) ", may have the possibility to be spoken by the user 
5 are managed, and only a command to be spoken by the user to 
activate the traffic information application not currently 
activated is managed. 

Fig. 20 is an example of a screen 201 displayed when 
the user has spoken "activate traffic information 

10 application" in the state of Fig. 18, and of a list 202 of 
the global commands receivable in this activated state. When 
the traffic information application enters an "interacting" 
state at the same time as its activation, the navigation 
application may transfer from "interacting" state to 

15. "activated" state. Then a group of the global commands as 
shown in Fig. 20 may become receivable. 

The applications may not need to automatically enter 
the "interacting" state when activated, but may transfer to 
the "activated" state. For example, during the procedure 

20 from Fig. 18 to Fig. 20, there is a change that the traffic 
information application is displayed on the screen. It is 
possible that the traffic- information application may 
transfer to the "activated" state and the navigation 
application may continue to be in the "interacting" state. 

25 This is because an operation to be next executed by the user 
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according to information obtained from the traffic 
information application may be predicted as one to be 
executed by the navigation application, such as renewed 
search for routes and search for bypasses. So the traffic 
5 information application having small possibility to 

interact with the user transfers to the "activated'' state 
and the navigation application continues in an 
"interacting" state, increasing the handling of the user. 

Generally, the application whose main purpose is to 

10 unilaterally provide information to the user and which does 
not interact with the user may transfer to the "activated" 
state, instead of the "interacting" state, even when called 
by the user by use of the global command, so that the user 
may naturally return to the flow of the interaction before 

15 calling the application. 

According to the present invention, in a car 
navigation system where many applications simultaneously 
exist, an optimum global command may become valid according 
to a state of each application, e.g. , to decrease the wrong 

20 recognition ratio, avoid the malfunctions, and execute a 
plurality of the applications by use of one speech, 
achieving a convenient user interface. 

As mentioned above a feature of the present invention 
is to provide a user-friendly interaction type user 

25 interface unit for increasing a recognition ratio in the 
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above-described systems where many speech interaction 
applications are managed. 

IN conclusion one example outline of the present 
invention disclosed herein to solve the above-described 
5 problems is as follows. 

A terminal of the present invention includes, at 
least, a control unit for controlling a plurality of 
applications, a speech input unit, a speech recognition 
engine for recognizing an input via the speech input unit, 

10 and a record unit for recording global commands which 

achieve interaction with a plurality of the applications 
other than an application currently interacting with a user. 
The control unit, when results of the speech recognition 
match the global commands, executes applications 

15 corresponding to the global commands, and manages the global 
commands in response to a state of each of the applications 
to solve the above-described problems. 

In concluding, reference in the specification to "one 
embodiment", "an embodiment", "example embodiment", etc., 

20 means that a particular feature, structure, or 

characteristic described in connection with the embodiment 
is included in at least one embodiment of the invention. The 
appearances of such phrases in various places in the 
specification are not necessarily all referring to the same 

25 embodiment. Further, when a particular feature , structure, 
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or characteristic is described in connection with any 
embodiment or component, it is submitted that it is. within 
the purview of one skilled in the art to effect such feature, 
structure, or characteristic in connection with other ones 
5 of the embodiments and/or components. Furthermore, for ease 
of understanding, certain method procedures may have been 
delineated as separate procedures; however, these 
separately delineated procedures should not be construed as 
necessarily order dependent in their performance, i.e., 

10 some procedures may be able to be performed in an alternative 
ordering, simultaneously, etc. 

This concludes the description of the example 
embodiments. Although the present invention has been 
described with reference to a number of illustrative 

15 embodiments thereof, it should be understood that numerous 
other modifications and embodiments can be devised by those 
skilled in the art that will fall within the spirit and scope 
of the principles of this invention. More particularly, 
reasonable variations and modifications are possible in the 

20 component parts and/or arrangements of the subject 

combination arrangement within the scope of the foregoing 
disclosure, the drawings and the appended claims without 
departing from the spirit of the invention. In addition to 
variations and modifications in the component parts and/or 
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arrangements, alternative uses will also be apparent to 
those skilled in the art. 
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